U.S. patent application entitled “TOOL FOR OPTIMIZING SYSTEM PERFORMANCE AND METHODS RELATING TO SAME”, filed Dec. 30, 2004 to the same inventor is hereby incorporated herein by reference.
1. Field of the Invention
Embodiments of the invention described herein pertain to the field of computer software. More particularly, but not by way of limitation, one or more embodiments of the invention enable efficient communication with a database comprising binary large object (BLOB) data.
2. Description of the Related Art
The primary use of database systems is to function as a mechanism for users to store and retrieve data. Hence most database systems act as a repository for various types of data including characters, numbers and other basic data types. In addition, data types including objects, images, video and other types of data are supported by most existing databases.
Within the last several years, as hard drive capacities and network bandwidth have continually increased, it has become significantly more common than was previously the case for database users to store and retrieve large data objects. Storing and retrieving such large data objects (e.g., image and video objects) can inhibit the efficiency of a DBMS system and reduce overall system performance. This is because most database systems are not designed to handle widespread access to large data objects and hence there is a significant burden placed on the system when access to these large data objects becomes a regular part of day to day operations. This reduced performance during storage and retrieval of large objects is caused by a number of different factors.
When a large data object is retrieved by a client the database reads the entire object with one read operation. This retrieval operation in turn causes the database to allocate a segment of memory on the database server in which to read the entire data object being retrieved. On a small scale such allocations can reduce performance, but do not necessarily cause a substantial drop in overall system performance. However, when such requests become more common and many users across the entire system request large objects, systems are required to handle many threads asking for similar functions thus causing a significant reduction in system performance. If for instance, 30 users initiate requests to retrieve different PDF objects where each object is approximately 100 mb in size, then the server allocates approximately 3 Gb of memory. In many cases the occurrence of such an allocation requirement will impact system performance.
The impact of retrieving and storing large data objects on memory occurs when a DBMS is asked to take other actions at the direction of the client (e.g., insert, update, etc. . . ). Microsoft SQL Server, for instance, typically allocates 4 times the amount of memory of the object to be inserted. So in cases where a 50 MB object is to be inserted the server allocates approximately 200 MB of memory to the insert task.
Another problem that occurs when large data objects are transmitted between a client and server is that the transmission of such objects causes an increase in the number of network collisions which in turn places a noticeable burden on the network and reduces overall system efficiency.
To alleviate the burdens placed on a system when utilizing large blocks of data a technique known as “blob chunking” is used to read smaller blocks of data from a BLOB field until the entire BLOB field is read. Blob chunking may also be used to write a series of smaller blocks of data to a BLOB field until the entire BLOB field is written. To date there has been no way to intelligently determine the size of blocks to break a read or a write of a BLOB field into as current attempts at BLOB chunking merely attempt to allow a system to operate without returning an “out of memory” error for example.
Because of the limitations described above there is a need for a system that allows greater efficiency in situations where large data objects are transferred between a server and a client.